arXiv:1507.06562vl [cs.NI] 23 Jul 2015 


To HTTP/2, or Not To HTTP/2, That Is The Question 


Matteo Varvello*, Kyle Schompt, David Naylor^, Jeremy Blackburn*, 
Alessandro Finamore*, Kostantina Papagiannaki* 

*Telef6nica Research, ^Case Western Reserve University, ^Carnegie Meiion University 


ABSTRACT 

As of February, 2015, HTTP/2, the update to the 16-year-old 
HTTP 1.1, is officially complete. HTTP/2 aims to improve 
the Web experience by solving well-known problems (e.g., 
head of line blocking and redundant headers), while intro¬ 
ducing new features (e.g., server push and content priority). 
On paper HTTP/2 represents the future of the Web. Yet, it 
is unclear whether the Web itself will, and should, hop on 
board. To shed some light on these questions, we built a 
measurement platform that monitors HTTP/2 adoption and 
performance across the Alexa top 1 million websites on a 
daily basis. Our system is live and up-to-date results can be 
viewed at III. In this paper, we report our initial findings 
from a 6 month measurement campaign (November 2014- 
May 2015). We find 13,000 websites reporting HTTP/2 
support, but only 600, mostly hosted by Google and Twit¬ 
ter, actually serving content. In terms of speed, we find no 
significant benefits from HTTP/2 under stable network con¬ 
ditions. More benefits appear in a 3G network where current 
Web development practices make HTTP/2 more resilient to 
losses and delay variation than previously believed. 

1. INTRODUCTION 

HTTP/2 (H2 for short) is the new version of HTTP, ex¬ 
pected to replace version 1.1 (HI) which was standardized 
in 1999. H2 promises to make the Web faster and more 
efficient by compressing headers, introducing server push, 
fixing head of line blocking, and loading page elements in 
parallel over a single TCP connection (cf. Section |2]i. Al¬ 
though the IETF does not require encrypting H2 connections 
with TLS (as originally proposed), the major browser ven¬ 
dors have decided to only support encrypted H2 traffic. 

While on paper H2 represents the future of the Web, it 
is unclear whether its adoption will face a struggle similar 
to IPv6. As discussed in Q, the adoption of a new pro¬ 
tocol in presence of ossification largely depends on the ra¬ 
tio between its benefits and its costs. Modern websites are 
designed to deal with Hi’s inefficiencies, employing hacks 
like spriting, inlining, etc. Ell. While H2 would remove 
the need for such hacks, in theory simplifying Web develop¬ 
ment, it is unclear how much performance improvement it 
can provide compared to HI coupled with the hacks above. 
Furthermore, given the large adoption of such hacks, it is un¬ 
clear if/how they can coexist with H2 (vital for the adoption 


of H2 as one cannot expect web developers to rebuild their 
websites overnight). 

Motivated by the above, in this work we build a measure¬ 
ment platform which monitors H2 adoption and assesses its 
performance. Our measurement platform consists of a mas¬ 
ter and several workers. The master is a Linux server that 
coordinates measurements and collect results. The workers 
are both PlanetLab machines and machines within our lab. 
We use PlanetLab to probe Alexa’s top 1 million websites 
daily looking for claims of H2 support. Then, we use our 
own machines to verify whether sites claiming H2 support 
actually do and, if so, to analyze which features they use and 
end-user experience. Results are published daily at IT]. 

This paper reports findings from a 6 months measurement 
campaign, from November 2014 until May 2015 (cf. Sec- 
tionlDi. Overall, we find that 13,000 websites report H2 sup¬ 
port, but only about 600, mostly hosted by Google and Twit¬ 
ter, actually serve content over H2 on a consistent basis. A 
macroscopic analysis of such content suggests that there was 
no significant content modification as websites adopt H2. A 
more in-depth analysis also unveils that classic HI hacks are 
still present with H2. For example, inlining (putting CSS 
styles and JavaScript code directly in HTML) is still widely 
used, reducing caching benefits at the browser; the same is 
true of domain sharding (spreading Web objects across sev¬ 
eral domains either on purpose or because needed) which 
causes only a 15% reduction of TCP connections used with 
H2. Interestingly, domain sharding makes H2 more resilient 
to losses and delay variation typical of mobile networks. 

2. BACKGROUND AND REUATED WORK 

HI is w ASCII protocol that allows a client to request/submit 
content from/to a server. HI is mostly used to fetch web 
pages, where clients request objects from a server and the re¬ 
sulting response is serialized over a persistent TCP connec¬ 
tion. Pipelining requests improves page load time for pages 
with many objects, but the benefits are limited since HI re¬ 
quires servers to serve requests in order. Thus, an early re¬ 
quest for a large object can delay all subsequent pipelined re¬ 
quests (head of line blocking). Clients mitigate this by open¬ 
ing several concurrent TCP connections to a server, which 
incurs additional overhead (TCP state on the server, TCP 
handshake latency, and TLS session setup in the case of 
HTTPS HSl). Accordingly, browser vendors set a limit on 
the maximum number of open connections to a domain, e.g.. 
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6 in Chrome and 15 in Firefox ll24ll . Web developers have 
dealt with this limitation through the use of domain shard- 
ing, where content is distributed across different domains, 
circumventing the single domain connection limit. Finally, 
HI requires the explicit transmission of headers on a per 
request/response basis. Therefore, common headers (like 
server version) are retransmitted with each object, causing 
high overhead especially for pages with a large number of 
small embedded objects. 

SPDY is a Google protocol for delivering web content. It 
is a fully binary protocol, enabling efficient parsing, lighter 
network footprint, and, most importantly, reducing exposure 
to security issues resulting from the failure to properly san¬ 
itize input values. SPDY opens a single TCP connection 
to a domain and multiplexes requests and responses, called 
streams, over that connection, which reduces TLS/SSL over¬ 
head at the client and load at the server. SPDY additionally 
introduces the concept of priority, which allows the client 
to load important objects (e.g., CSS and JavaScript) earlier, 
server push, which allows the server to push objects before 
the client requests them, and header compression, which re¬ 
duces redundant information shared between objects. 

Many preliminary studies investigate SPDY performance, 
but they fall short either because they focus on an handful 
of websites ca, or explore a limited number of parame¬ 
ters DiiiaGa. Although the results of these studies are 
mostly contradictory, they indicate poor SPDY performance 
on mobile networks. Motivated by this observation, Erman 
et al. 10 set up SPDY and HI proxies within a 3G network, 
and monitored performance while fetching Alexa’s top 20 
websites They find that SPDY performs poorly in mobile 
networks due to many wasted retransmissions. 

To address difficulties in measuring SPDY performance, 
Xiao et al. ll25l devise a measurement platform to explore 
SPDY performance at a microscopic level. They find that 
SPDY tends to outperform HI, in the absence of browser de¬ 
pendencies and computation, however the gains are reduced 
when both browser dependencies and computation are con¬ 
sidered (with the caveat that server push can squeeze addi¬ 
tional performance gains from SPDY). 

In our work we also build a measurement platform and 
conduct experiments on 3G, but we focus on H2. More im¬ 
portantly, we do not need proxies since we target real servers 
with H2 support, leading to different conclusions than 0. 

H2 builds on SPDY, though there are a few differences be¬ 
tween them. For example, H2 uses HPACK IT^ for header 
compression, eliminating SPDY vulnerability to the “crime" 
attack na, and some framing changes make H2 more reg¬ 
ular than SPDY. De Saxce et al. quantified H2 performance 
in ll20ll . Using the top 20 Alexa websites, but stripping im¬ 
portant components like the presence of multiple domains, 
they investigate H2 performance while controlling network 
delay, bandwidth, and losses via a measurement platform 
built with an open-source client and server. Their results 
confirm previous observations about SPDY in ll25l . We also 
investigate H2 performance but we use 600 real, unmodified 


H2 servers discovered by monitoring the top 1 million Alexa 
sites, allowing us to gain insights on H2 performance in the 
wild without ignoring widely used HI optimizations like do¬ 
main sharding and inlining. 

NPN and ALPN The Next Protocol Negotiation (NPN) 0, 
an extension to Transport Layer Security (TLS) proposed by 
Google, is currently used to negotiate SPDY as an applica¬ 
tion level protocol over port 443 and to perform SPDY ver¬ 
sion negotiation. However, it is not SPDY specific; servers 
supporting NPN report which application layer protocols they 
handle, e.g., H2 or SPDY, and the client then selects which 
protocol to use. This negotiation is done during the TLS 
handshake without additional round trips. The Application- 
Layer Protocol Negotiation (ALPN) ll22l extension to TLS 
is an IETF proposed standard set to supersede NPN. 

3. MEASUREMENT PLATFORM 

This section describes our measurement platform, along 
with a collection of tools we deployed for data collection. 

3.1 Tools 

prober is a lightweight bash script that identifies which pro¬ 
tocols a website announces. To do so, prober attempts an 
NPN negotiation and reports either the list of protocols an¬ 
nounced by the server or failure. NPN negotiation is per¬ 
formed using OpenSSL ifTTIl . Next, prober checks for H2 
support over cleartext by including an UPGRADE header in 
an HI request. Finally, prober checks if the site supports 
QUIC, a Google transport protocol designed for SPDY ifT^ . 
QUIC support is advertised with an Alternate- Protocol header 
in any response; we look for this header in the response to 
our upgrade request. 

h2-lite is a tiny H2 client that attempts to download the root 
object of a website using H2. H2-lite uses the H2 library im 
implemented in Node.js 0. H2-lite will follow any redi¬ 
rects to obtain the root object and report any protocol errors 
encountered along the way. 

h2-full is a headless browser that supports H2, SPDY, and 
HI. H2-full uses the Zombie.js headless browser [|2l in con¬ 
junction with the H2 library described above, a SPDY li¬ 
brary 0 implemented in Node.js, and the built-in Node.js 
HI library. H2-full can parse a website’s root object, re¬ 
cursively fetch embedded objects, and execute JavaScript. 
However only a single protocol per run can be used, a limita¬ 
tion further discussed in Section [33] H2-full reports metrics 
like page load time, number of objects downloaded, number 
of domains present, number of TCP connections, and object 
sizes. We also collect all HTML for offline analysis, 
chrome-loader is a tool written in Python that loads pages 
using Chrome. It extracts object size and timing informa¬ 
tion using chrome-har- capturer 0. Unfortunately, SPDY 
and H2 cannot be enabled/disabled independently, and since 
Chrome does not programmatically report which protocol 
it used, we know only that H2 or SPDY was used, but not 
which. 

page-saver is a bash script that automates a browser’s “Save 
Page As" feature 0. We use page-saver to collect the ground 
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truth of a web-page’s composition, and compare it, when 
possible, with h2-full and chrome-loader. Though any browser 
can be coupled with page-saver, we use Chrome for consis¬ 
tency with the chrome-loader. 

3.2 Description 

Our measurement platform consists of a master and sev¬ 
eral workers. The master issues crawl requests to the work¬ 
ers, which are deployed both in PlanetLab and in machines 
in our labs (both U.S. and Spain). We use PlanetLab for sim¬ 
ple measurements at large-scale, and our lab machines for 
more complex measurements where reliable machines guar¬ 
antee higher accuracy in data collection. Lab machines are 
also useful when physical access to a machine is needed, 
e.g., to install a 3G dongle. Using a distributed system like 
PlanetLab ensures that our measurements are not (overly) 
biased by the geographic distance between clients and the 
HTTP servers they are accessing. The master constantly 
monitors PlanetLab to identify a pool of “reliable” machines 
(at least 500 MB of free memory, CPU load under 30%, and 
no network connectivity issues). 

Our measurement campaign consists of three phases, which 
we describe here in detail. 

Phase I: First, the master launches an instance of the prober 
at each reliable PlanetLab worker. Each worker is then as¬ 
signed an unique 100-website list to probe. Once a worker 
has probed each website on its list, it reports results to the 
master, which stores them in a local database, and obtains 
a new list if one is available. This approach ensures load 
balancing among workers; faster workers tend to complete 
more tasks. To deal with stragglers (slow workers), the mas¬ 
ter re-assigns uncompleted tasks to new workers after a time¬ 
out T set as the average task completion time across workers. 
Phase I terminates when we have determined which applica¬ 
tion layer protocols are announced by the Alexa top IM. 
Phase II: When Phase I terminates, the master extracts the 
list of websites announcing H2 support. Then, it launches 
an instance of h2-lite to fetch the root object from each 
website. When Phase II terminates, the local database now 
includes a list of sites that actually serve content via H2. 

Because the H2 library requires more up-to-date software 
than is available on PlanetLab, we run Phase II on 3 servers 
under our control—2 in Barcelona, Spain and 1 in Cleve¬ 
land, USA. On each, we limit the number of concurrent web¬ 
site fetches to 10. Critically, Phase I limits the number of 
websites that must be explored in Phase II to a small enough 
number so that our 3 servers can run Phase II daily. 

Phase III: Once per week. Phase III runs on the list of web¬ 
sites that Phase II has conhrmed to deliver content over H2. 
Each website is fetched 3 times each with H2, HI, and SPDY 
(only if the website declared support in Phase I). Phase III is 
run entirely from a single server in Barcelona, Spain for uni¬ 
formity of measurement apparatus across all websites and 
the website fetches are run sequentially to limit the impact 
of network load on the results. 

3.3 Limitations 

The main limitation of our measurement platform is the 


lack of support for ALPN. This is due to the fact that ALPN 
support was added to OpenSSL only in version 1.0.2 re¬ 
leased on January 22nd, 2015. Also, running the most re¬ 
cent version of OpenSSL on PlanetLab is tedious, due to a 
diverse and old set of operating systems. 

In order to quantify the impact of this limitation, we in¬ 
strumented one of our lab machines to crawl the Alexa top 
1 million websites looking for ALPN support. We repeat 
this procedure once a month between Eebruary and April 
2015, with the goal to verify a potential switch from NPN 
to ALPN. This operation can be done from a single machine 
in a couple of days, without the need of the massive Planet¬ 
Lab scale. As of April 2015, only 0.5% of the Alexa top 1 
million websites support ALPN, and they all support NPN 
as well. We plan to keep monitoring ALPN and adapt our 
measurement platform in the future to make sure it can react 
quickly to ALPN eventual adoption. 

The Node.js SPDY and H2 libraries do not support falling 
back to another protocol if SPDY or H2 are not negotiated, 
respectively. Eurther, the SPDY library does not give any in¬ 
dication of which connection has failed when SPDY cannot 
be negotiated. Thus, we are unable to fully download web¬ 
pages if there are embedded objects that are not accessible 
using the same protocol as the root object. In the evaluation, 
we discuss how frequently we run into this limitation and 
what counter-measures we take. 


4. RESULTS 

We leverage our measurement platform to collect traces 
between November 10th, 2014 and May 6th, 2015. This 
section summarizes the analysis of the traces collected. We 
invite the reader to access fresh data and analysis at IT). 

4.1 Adoption 

Over six months—and about 200 million NPN-like 
negotiations—we have recorded support for 44 applica¬ 
tion protocol versions, with SPDY accounting for 34 of 
them. Pig. |l(a)| shows the evolution over time of the frac¬ 
tion of websites reporting to support either http/1.1, 
spdy/3.1, or h2 (cf. Section|2]l, for which we distinguish 
between versions h2-14, h2-15, h2-l 6, and h2-17. 

Pig. |l(a)[ left) shows that, on November 10th 2014, about 
92k websites out of 1 million declared, via NPN, support for 
HI. This is an underestimate of HI usage since HI does 
not require NPN. This measure really reveals the number 
of websites supporting NPN. Accordingly the hgure shows 
that, over 5 months, NPN support grows by 20%, from 92k 
to about 110k websites. A similar trend is observable for 
SPDY, where the number of supporting websites goes from 
55k to 62k. This is not surprising since NPN and SPDY are 
very much coupled. However, in the last month we observed 
an opposite trend, with NPN support dramatically dropping 
below where it was in November 2014. We verihed that this 
trend is not related to our measurement platform by testing 
the set of websites that withdrew NPN support from two con¬ 
trolled machines. Note that the introduction of ALPN does 
not explain this trend either (cf. Section ITjT l. 
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Figure 1; H2 Adoption. 


Fig. |l(a)[ right) focuses on H2 adoption only. Overall, the 
plot shows about 13k websites declaring support, but neg¬ 
ligible growth over 6 months. Note that, to this date, not 
even a single website supports H2 over cleartext. In Novem¬ 
ber/December, most servers bounce back and forth between 
announcing version h2-14 and h2-15. The most interesting 
event happens between December 14th and 18th, when the 
number of servers supporting H2 drops from 13k to only 20 
websites. This happens because Google, the major supporter 
of H2 (cf. Fig. |l(c)[ ) disabled H2 support on their servers due 
to an issue with SDCH compressed content in Chromium^ 
It takes a month for the problem to be fixed and H2 support 
is re-enabled between January 11th and the 15th. Interest¬ 
ingly, it takes Google four days to apply this change to all 
its servers, probably reflecting a specific roll-out policy. Be¬ 
tween January and April 2015, H2 support is quite stable. 
Version h2-16 appears first on February 19th, but it is not 
used by most servers. The final H2 draft, h2-17, is released 
on February 11th, but support starts showing only around 
April 16th, reaching 131 websites as of May 6, 2015. 

We now focus on the 13k websites announcing H2 sup¬ 
port and check whether they actually serve content with it. 
Figure [T(b)| shows the same trend as in Fig. |l(a)[ with a large 
gap due to Google suspending H2 support temporally. The 
figure also shows that less than 5% of the websites actually 
serve content through H2; the remaining 95% return code 
301 (resource moved) and redirect H2 requests to HI. Only 
in the first month, between November 10th and December 
17th, is an actual growth of H2 support noticeable, with 120 
new websites starting to serve H2 content. None of these 
websites has withdrawn support for H2. 

Fig. I l(c)| summ arizes which organizations are behind 
H2 support. For a comparison, we also report the same in¬ 
formation for SPDY; out of the 50-60k websites announc¬ 
ing SPDY support, about 23k actually serve some content 
through it, if requested. Also, QUIC is available at each of 
these sites. Note that we do not report the evolution over 
time of SPDY announce and support (dual of Fig. |l(b)| i be¬ 
cause we started collecting this information in the end of 
April 2015, and thus no interesting pattern is visible yet. 
Overall, Fig. |l(c)| shows that only two major organizations 
support H2, Google and Twitter, with Google controlling 
about 99% of the websites. In comparison, the SPDY ecosys- 

'https://lists.w3.Org/Archives/Public/ietf-http-wg/20140ctDec/0960.html 


tern is more complex, comprising 131 different organiza¬ 
tions including CloudFlare and Facebook. 

According to Alexa, 50% of the websites supporting H2 
today are in the top 100k websites. Also, three of the top 10 
websites are present; google . com, twitter. com, and 
youtube . com. Thus, only a handful of sites support H2 
today, but those that do tend to be very popular. 

4.2 Content Analysis and Delivery 

Next we investigate the composition of SPDY and H2 en¬ 
abled sites (e.g.. How many objects do they embed? Do they 
exploit sharding and inlining?) and how content is served 
(e.g.. Do they use the same application protocol for all ob¬ 
jects? How many connections do they require?). For this 
analysis we focus on April 30, 2015. 

Number of objects per website Fig. |2(a)| shows the Cumu¬ 
lative Distribution Function (CDF) of the number of objects 
per webpage when accessing them from a PC (desktop user 
agent) and a 3G dongle (mobile user agent). Notice how 
different tools report very similar results in both scenarios. 
The only macroscopic difference is related to Zombie under¬ 
estimating the number of objects compared to both Chrome 
and page-saver for 40% of the websites. These sites have 
a very similar composition; through manual inspection, we 
find that Zombie fails to dynamically load objects created 
through JavaScript or CSS embedded objects (e.g., back¬ 
ground images). In particular, 70% of sites where Zombie 
misses at least 4 objects are the different country-specific 
Google homepages (e.g., google.es, google.jp). Such phe¬ 
nomenon is less visible in the mobile version where this be¬ 
havior is indeed less frequent. 

Contrarily, Chrome seems to over-estimate the number of 
objects for about 20% of the sites while both Zombie and 
page-saver see similar figures. This is curious since page- 
saver also uses Chrome. Most of the sites are *.appspot.com 
and contain media files (e.g., .ogg, .woff2, or .glsl) which 
Chrome does not export under the “Save Page As” function. 
Note that many of these files are loaded by JavaScript which 
explains why Zombie misses them as well. The takeaway is 
that web performance analysis is hard and using a collection 
of tools helps gather meaningful results. 

Overall, most websites are composed of a small number of 
objects, e.g., 80% of these sites only have about 20 objects, 
which reflects to an overall page of less than 1 MB. This is 
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Figure 2: Content analysis and delivery. 


smaller than what we observe on the Web today, where, on 
average, a page holds about 100 objects with an overall size 
of 2 MB ll2^ . This indicates that H2 today is mostly serving 
simple content, fewer in objects and size compared to the 
rest of the Web. 

Page composition Websites today try to optimize delivery 
using a combination of sharding. Mining, spriting and con¬ 
catenation. Unfortunately, we see no straightforward way to 
measure the latter two, so in this work we focus on the first 
two. Fig. |2(b)| shows the results of this analysis. 

Focusing first on sharding, the left boxplots in Fig. |2(b)[ left) 
report the distribution of the number of unique domains per 
site. Two observations hold. First, although the distributions 
for HI and H2 are similar, we found several instances where 
a few objects were not loaded via HI but were loaded by 
H2. We conjecture this has to do with geo-location; when 
accessing these sites via HI from Spain (but not the US) a 
few fewer objects are returned. Second, we see that the num¬ 
ber of domains per page has a median of 2, a mean of 3.71, 
and a top lOth-percentile of 7 for both HI and H2. Some 
sites will open upwards of 30 connections solely because of 
a large number of domains to contact. 

Next, in Fig. |2(b)( middle) we plot the distribution of the 
average number of objects per domain per site. We observe 
that most domains are used for relatively few objects (me¬ 
dian = 2, mean = 2.96, top 10th- percentile of 5.64, and a 
max of 44). This is further evidence of sharding-esque be¬ 
havior. Whether intentional or not, there are many sites that 
exhibit the behavior we would expect from sharding. This 
means many cuivent websites still require multiple TCP con¬ 
nections, even if they switch to H2. 

Finally, Fig. |2(b)( right) plots the size of inlined CSS/JS 
to the total size of the main HTML for each site. We hnd 
that inlined CSS/JS makes up over 80% of the HTML con¬ 
tent for over 40% of the sites under consideration. Although 
not shown in the plot, we further observed that CSS and 
JavaScript inlining is prevalent: about 70% of sites inline 
over 90% of their CSS and JavaScript objects. With HI, in¬ 
lining can help ensure the page loads faster by not requiring 
an extra TCP connection; however this is no longer an issue 
in H2, so these sites are (in theory) suffering a performance 
hit since inlined content cannot be cached separately by the 
client. 

Application protocol consistency Even if a website sup¬ 


ports SPDY or H2, embedded objects might be subject to 
different application protocol negotiation. Indeed, we find 
that while 75% of the sites serve 100% of their content with 
H2, only 25% serve the same amount with SPDY (and of 
course 100% of sites serve 100% of content with HI). This 
result can be either an artifact of the subset of sites we are 
investigating, chosen because of their H2 rather than SPDY 
support, or of the SPDY library used. 

Number of connections We analyze the number of TCP 
connections used to load websites entirely delivered through 
HI or H2. Fig. |2(c)| shows the CDF of the number of con¬ 
nections required by Zombie and Chrome, per protocol. We 
only differentiate between H2 and HI since we have no hner 
grained information for Chrome. No results are shown for 
page- saver since it does not report connection counts. In 
the desktop version, H2 with Zombie and H2 with Chrome 
have a similar trend with 78-81% of web-pages requiring 4 
or fewer connections. Differences in the number of objects 
per page between Zombie and Chrome (cf. Fig. |2(a)| i cause 
Chrome to use more connections. However, we also dis¬ 
covered an edge case in the H2 library which allows mul¬ 
tiple connections to the same domain upon multiple con¬ 
current requests for objects within the domain, explaining 
why Zombie rarely has more connections than Chrome. For 
HI, Chrome uses more connections for 55% of web- pages, 
again because of differences in the number of objects per 
page. The effect is more pronounced in HI because Chrome 
allows up to 6 concurrent HI connections per domain ll24l . 

In the mobile version, we observe a dramatic difference in 
behavior; Chrome requires fewer connections than Zombie 
across all protocols. First, Zombie requires more connec¬ 
tions in the mobile version than in the desktop because more 
objects are fetched in the mobile version (cf. Fig. |2(a)| i. Sec¬ 
ond, we suspect that Chrome is under-reporting connections 
because for 40% of web-pages. Chrome reported opening 
fewer connections than the number of domains which is, of 
course, impossibl^l Note that this under-reporting occurs 
in the desktop version as well and therefore our measure¬ 
ments of connections used by Chrome are a lower bound. If 
we now focus on differences between protocols measured by 
the same tool, we notice that H2 reduces the number of TCP 


^Chrome likely reuses connections from a previous download or 
preemptively opens connections before they are needed. 
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Figure 3: CDF of page load time. 


connections compared to HI by 10-15%. 

Stability over time Given the extensiveness of the analy¬ 
sis (6 months), we further crosschecked the variation over 
time of the effects described above. Specifically, we com¬ 
pute the cosine similarity for each metric discussed both 
between successive weeks and between different protocols. 
Apart from the number of connections, we find the cosine 
similarity always being between 0.9 and 0.98, i.e., no signif¬ 
icant content modification, at a macroscopic level, as web¬ 
sites start adopting H2. 

4.3 Performance 

We here investigate H2 speed in term of page load time 
(PLT). There is no standardized definition of PLT, so its mean¬ 
ing changes slightly between our tools. In Zombie, PLT 
indicates the time elapsed between sending the initial ob¬ 
ject request for a page and the “visited” event that signals 
Zombie has finished the page. Note that not all of a page’s 
objects must be downloaded before a website is considered 
“finished.” Similarly, PLT may be measured from Chrome 
via the JavaScript “onLoad” event which indicates that the 
page’s resources have been loaded. Again, the “onLoad” 
event may occur before all objects within the page are down¬ 
loaded, since it does not apply to objects loaded by scripts. 

Because we measured no major content modification over 
6 months (cf. Section |4^ . we focus on a single data point 
(April 30, 2015), differentiating between “desktop” (fiber H- 
desktop user-agent) and “mobile" (3G H- mobile user-agent). 
Desktop and mobile data are collected from our lab in Spain, 
using both Zombie and Chrome. Because only few of the 
sites we are monitoring fully support SPDY, we focus the 
performance analysis on H2 and HI only. Also, we consider 
only the subset of sites that can serve 100% of their content 
through H2 (477 desktop and 471 mobile). 

Figure Oleft) shows the CDF of the average PLT (three 
downloads per website) for the desktop scenario. Chrome 
reports a negligible difference between PLT values obtained 
with HI and H2 (apart from the 10% of websites with PLT 
> 2 seconds) where HI saves on average ^200 ms. If we fo¬ 
cus on Zombie results, we notice H2 being faster (between 
100 and 500 ms) for half of the sites whose PLT values fall 
between the 40th and 90th percentile. Note also that Zombie 
shows much higher PLT values compared to Chrome, prob¬ 
ably due to Zombie being far less optimized than Chrome. 

Next we consider PLT for mobile sites. Figure Oright) 



Figure 4: PLT under controlled network configurations. 

shows that H2 consistently outperforms HI (according to 
both Chrome and Zombie). This result is in contrast with ISl . 
which suggests that SPDY, which is the basis of H2, pays a 
large penalty in mobile. We do not observe this behavior 
because our experiments do not require an H2 proxy, since 
we use several real-world, deployed servers. Accordingly, 
we do not force a single TCP connection as in IHl, but open 
as many connections as needed (cf. Fig. |2(c)[ ), which is 
a side effect of spreading content across multiple domains 
(cf. Fig. |2(b)| l. It follows that domain sharding makes H2 — 
and likely SPDY—more resilient to losses and delay varia¬ 
tion typical of mobile networks. 

To further understand the previous results, we repeat the 
desktop scenario above —Zombie only—while controlling 
network bandwidth, delay, and loss. FigH] shows the PLT 
distributions for both HI and H2, for several combinations 
of network parameters. We start by limiting the available 
bandwidth to 1 Mbps; compared to the scenario with no 
bandwidth limitation (30 Mbps as measured locally), we no¬ 
tice HI suffering a higher penalty in PLT compared to H2. 
A similar trend is noticeable as we create more challenging 
network conditions, such as additional network delay and 
packet losses (up to 100 ms and 0.5%). Finally, in the last 
scenario we limit the available bandwidth to only 1 Mbps, 
and increase the packet losses to 2%, emulating a plausible 
mobile scenario. The results confirm what was previously 
observed in Fig. [3 with H2 greatly improving PLT com¬ 
pared to HI. 

5. CONCLUSION 

This work presents a measurement platform to monitor 
both adoption and performance of H2, the recently standard¬ 
ized update of HI. On a daily basis, our platform checks for 
which protocols the top 1 million Alexa websites announce 
support. Next, it checks which sites actually support H2. 
Finally, once a week we benchmark H2 performance using 
the websites previously discovered. Results are available at 
IT] and are updated daily. In this paper, we report our ini¬ 
tial findings from the data collected between November 2014 
and May 2015. We find 13,000 websites already announc¬ 
ing H2 support, out of which 600 websites (mostly hosted 
by Google and Twitter) deliver content over H2. A more 
in-depth analysis also unveils that classic HI hacks are still 
present with H2, and in challenging environments like 3G 
mobile, might result in performance benefits. 


6 


















6. REFERENCES 

[1] Is the web http/2 yet? 

http://isthewebhttp2yet.com 

[2] Node.js. https ://node js . org/, 

[3] Adam Langley. Tls next protocol negotiation, 
http://tinyurl.com/d5owb9g 

[4] Akhshabi, S., and Dovrolis, C. The Evolution 
of Layered Protocol Stacks Leads to an 
Hourglass-shaped Architecture. In Proc. ACM 
SIGCOMM (Toronto, Ontario, Canada, Aug. 2011). 

[5] Andrea Cardaci. Chrome har capturer. 
http://tinyurl.com/boajgjw 

[6] Anurag Biyani. Automate save page as. 
http://tinyurl.com/ph5u3mr 

[7] Assaf Arkin. Zombie.js: insanely fast, headless 
full-stack testing using node.js. 

http://zombie.js.org/ 

[8] Erman, J., Gopalakrishnan, V., Jana, R., and 
Ramakrishnan, K. Towards a SPDYier Mobile 
Web? In Proc. ACM CoNEXT (Santa Barbara, CA, 

Dec. 2013). 

[9] Eedor Indutny. node-spdy. 

https://github.com/indutny/node-spdy 

[10] G. White and J-E. Mule and D. Rice. Analysis 
of spdy and tcp initcwnd.. 

http://tinyurl.com/o7hoy13 

[11] Gabor MolnAr. node-http2. 

https://github.com/moInarg/node-http2 

[12] Google. Quic, a multiplexed stream transport over 
udp. https://WWW.chromium.org/quic 

[13] Google. Spdy whitepaper. 

http://tinyurl.com/ykyzrur 

[14] Guy Podjarny. Not as spdy as you thought, 
http://tinyurl.com/p37msqz 


[15] J. Rizzo and T. Duong. The crime attack. In 
Ekoparty, 2012. 

[16] Naylor, D., Einamore, A., Leontiadis, L, 
Grunenberger, Y., Mellia, M., Munafo, M., 
Papagiannaki, K., and Steenkiste, P. The Cost 
of the “S” in HTTPS. In Proc. ACM CoNEXT 
(Sydney, Australia, Dec. 2014). 

[17] OpenSSL. Openssl; The open source toolkit for 
ssl/tls. https : //www. openssl. org/ 

[18] Padhye, j., and Nielsen, H. E. A comparison of 
spdy and http performance. Tech, rep., 2012. 

[19] Roberto Peon and Herve Ruellan. Hpack - 
header compression for http/2. 

http://tinyurl.com/qjmmgxo 

[20] Saxce, H. D., Oprescu, L, and ChenSaamer, 

Y. Is HTTP/2 Really Paster Than HTTP/Ll? In Proc. 
IEEE Global Internet Symposium (GI) (Hong Kong, 
CH, Apr. 2014). 

[21] Stenberg, D. http2, background, the protocol, the 
implementations and the future. 
http://daniel.haxx.se/http2/http2-vL9.pdf. 

[22] Stephan Priedl, Andrei Popov, Adam 
Langley, Emile Stephan. Transport layer security 
(tls) application-layer protocol negotiation extension. 
https://tools.ietf.org/html/rfc7301 

[23] The http archive. 

http://httparchive.org 

[24] Tuan, N. A. Maximum concurrent connections to the 
same domain for browsers. 

http://tinyurl.com/q62vzzs 

[25] Wang, X. S., Balasubramanian, A., 
Krishnamurthy, a., and Wetherall, D. How 
speedy is spdy. In Proc. NSDI (Seattle, WA, Apr. 
2014). 


7 


